Simultaneous Perturbation Particle Swarm Optimization and Its FPGA Implementation

The particle swarm optimization technique is one of the promising tools to find a proper optimum for an unknown function optimization. Especially, global search capability of the method is very powerful. The particle swarm optimization utilizes common knowledge of the group and individual experiences effectively. That is, direction for the best estimator that a particle has ever reached, direction for the best one that all particles have ever found and momentum are successfully combined to determine the next direction. At the same time, the method does not utilize gradient of the objective function. Only values of the objective function are used. In many applications, it is difficult or impossible to obtain the gradient of an objective function. Then, the particle swarm optimization can take advantage of the merit. However, this means that the method does not use local information of the function. Even if a particle is close to a global optimal, the particle moves based on three factors described above. In this case, it seems better to search neighbour area carefully. To do so, local information such as gradient is necessary. On the other hand, the simultaneous perturbation method is a kind of stochastic gradient method. The scheme can obtain the local information of the gradient without direct calculation of the gradient. The simultaneous perturbation estimates the gradient using a kind of finite difference technique. However, even if dimension of the parameters are large, the simultaneous perturbation requires only two values of the target function. Therefore, we can apply this to high dimensional optimization problems in effect. As mentioned now, since the simultaneous perturbation is a stochastic gradient method, we cannot expect global search capability. That is, this method cannot give a global optimal but a local one. Combination of the particle swarm optimization and the simultaneous perturbation optimization will yield interesting algorithms which have advantages of these two approaches. There are some ways to combine the particle swarm optimization and the simultaneous perturbation method. In this paper, we propose four new algorithms based on combinations of the particle swarm optimization and the simultaneous perturbation. Some results for test functions are also shown. Moreover, hardware implementation of these kinds of algorithms is interesting research target. Especially, the particle swarm optimization has plural search points which are